When I use the airport utility to edit my base station configuration it doesn't prompt me for the base station password ... does this mean anyone can edit my airport extreme without knowing and supplying the password?
This all depends on whether you are using the same password for the Base Station and the wireless network.
If you are using the same Base Station password as the Wireless network is using, then any user who is connected to your wireless will be able to access settings in AirPort Utility, if they have it installed on their device.
IF you are using a different Base Station password than the wireless network, then users on the wireless will not be able to access the settings using AirPort Utility.
The big question to ask here.......do you remember setting up separate Base Station and Wireless passwords, or did you only set up one password?
If you only set up one password, then now would be a very good time to set up a Base Station password for your AirPort router.